Papers with large generative language models
RobustQA: Benchmarking the Robustness of Domain Adaptation for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
Rujun Han, Peng Qi, Yuhao Zhang, Lan Liu, Juliette Burger, William Yang Wang, Zhiheng Huang, Bing Xiang, Dan Roth
| Challenge: | Existing ODQA datasets consist mainly of Wikipedia corpus, and are insufficient to study models’ generalizability across diverse domains. |
| Approach: | They propose a benchmark to evaluate ODQA's domain robustness using Wikipedia corpus . they annotate QA pairs in retrieval datasets with rigorous quality control . |
| Outcome: | The proposed benchmark improves model performance on annotated QA pairs in retrieval datasets with rigorous quality control. |
MedQA-SWE - a Clinical Question & Answer Dataset for Swedish (2024.lrec-main)
Copied to clipboard
| Challenge: | MedQA-SWE is a clinical question & answering dataset in Swedish . it was created from exams aimed at evaluating doctors’ clinical understanding and decision making . |
| Approach: | They propose to create a multiple choice, clinical question & answering (Q&A) dataset in Swedish consisting of 3,180 questions. |
| Outcome: | The proposed dataset includes 3,180 questions and is the first open-source clinical Q&A dataset in Swedish. |